2023-12-28 11:43:54,971 [ 277954 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/_temp/test/git-repo-copy (runner:42, check_args_and_update_paths)
2023-12-28 11:43:54,971 [ 277954 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration (runner:90, check_args_and_update_paths)
2023-12-28 11:43:54,971 [ 277954 ] INFO : src dir is not set. Will use /home/ubuntu/_work/_temp/test/git-repo-copy/src (runner:97, check_args_and_update_paths)
2023-12-28 11:43:54,972 [ 277954 ] INFO : base_configs_dir: /home/ubuntu/_work/_temp/test/git-repo-copy/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration  (runner:99, check_args_and_update_paths)
2023-12-28 11:43:54,972 [ 277954 ] INFO : Unknown image altinityinfra/dotnet-client (runner:357, <module>)
Running pytest container as: 'docker run   --rm --name clickhouse_integration_tests_hcivdn --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/src/Server/grpc_protos:/ClickHouse/src/Server/grpc_protos --volume=/run:/run/host:ro --mount type=bind,source=/home/ubuntu/_work/_temp/test/dockerd_volume_dir,target=/var/lib/docker -e DOCKER_HELPER_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_BASE_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_KERBERIZED_HADOOP_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_KERBEROS_KDC_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_MYSQL_JAVA_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_MYSQL_JS_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_MYSQL_PHP_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_NGINX_DAV_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc   -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600  -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 5 -rfEps --run-id=1 --color=no --durations=0 'test_system_merges/test.py::test_mutation_simple[]' 'test_system_merges/test.py::test_mutation_simple[replicated]'  -vvv" altinityinfra/integration-tests-runner:0-57762a45dd8d7579a37ae9643654cde6896312dc '.
Start tests
============================= test session starts ==============================
platform linux -- Python 3.10.12, pytest-7.4.3, pluggy-1.3.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /ClickHouse/tests/integration
configfile: pytest.ini
plugins: timeout-2.2.0, anyio-4.2.0, repeat-0.9.3, order-1.0.1, xdist-3.5.0, random-0.2
timeout: 900.0s
timeout method: signal
timeout func_only: False
created: 5/5 workers
5 workers [2 items]

scheduling tests via LoadFileScheduling

test_system_merges/test.py::test_mutation_simple[] 
[gw1] [ 50%] FAILED test_system_merges/test.py::test_mutation_simple[] 
test_system_merges/test.py::test_mutation_simple[replicated] 
[gw1] [100%] FAILED test_system_merges/test.py::test_mutation_simple[replicated] 

=================================== FAILURES ===================================
____________________________ test_mutation_simple[] ____________________________
[gw1] linux -- Python 3.10.12 /usr/bin/python3

started_cluster = <helpers.cluster.ClickHouseCluster object at 0x7f2592952a40>
replicated = ''

    @pytest.mark.parametrize("replicated", ["", "replicated"])
    def test_mutation_simple(started_cluster, replicated):
        clickhouse_path = "/var/lib/clickhouse"
        db_name = "test"
        table_name = "mutation_simple"
        name = db_name + "." + table_name
        table_path = "data/" + db_name + "/" + table_name
        nodes = [node1, node2] if replicated else [node1]
        engine = (
            "ReplicatedMergeTree('/clickhouse/test_mutation_simple', '{replica}')"
            if replicated
            else "MergeTree()"
        )
        node_check = nodes[-1]
        starting_block = 0 if replicated else 1
    
        try:
            for node in nodes:
                node.query(
                    f"create table {name} (a Int64) engine={engine} order by tuple()"
                )
    
            node1.query(f"INSERT INTO {name} VALUES (1), (2), (3)")
    
            part = "all_{}_{}_0".format(starting_block, starting_block)
            result_part = "all_{}_{}_0_{}".format(
                starting_block, starting_block, starting_block + 1
            )
    
            # ALTER will sleep for 3s * 3 (rows) = 9s
            def alter():
                node1.query(
                    f"ALTER TABLE {name} UPDATE a = 42 WHERE sleep(9) = 0",
                    settings=settings,
                )
    
            t = threading.Thread(target=alter)
            t.start()
    
            # Wait for the mutation to actually start
            assert_eq_with_retry(
                node_check,
                f"select count() from system.merges where table='{table_name}'",
                "1\n",
                retry_count=30,
                sleep_time=0.1,
            )
    
>           assert (
                split_tsv(
                    node_check.query(
                        """
                SELECT database, table, num_parts, source_part_names, source_part_paths, result_part_name, result_part_path, partition_id, is_mutation
                    FROM system.merges
                    WHERE table = '{name}'
            """.format(
                            name=table_name
                        )
                    )
                )
                == [
                    [
                        db_name,
                        table_name,
                        "1",
                        "['{}']".format(part),
                        "['{clickhouse}/{table_path}/{}/']".format(
                            part, clickhouse=clickhouse_path, table_path=table_path
                        ),
                        result_part,
                        "{clickhouse}/{table_path}/{}/".format(
                            result_part, clickhouse=clickhouse_path, table_path=table_path
                        ),
                        "all",
                        "1",
                    ],
                ]
            )
E           assert [] == [['test', 'mutation_simple', '1', "['all_1_1_0']", "['/var/lib/clickhouse/data/test/mutation_simple/all_1_1_0/']", 'all_1_1_0_2', '/var/lib/clickhouse/data/test/mutation_simple/all_1_1_0_2/', 'all', '1']]
E             Right contains one more item: ['test', 'mutation_simple', '1', "['all_1_1_0']", "['/var/lib/clickhouse/data/test/mutation_simple/all_1_1_0/']", 'all_1_1_0_2', ...]
E             Full diff:
E               [
E             +  ,
E             -  ['test',
E             -   'mutation_simple',
E             -   '1',
E             -   "['all_1_1_0']",
E             -   "['/var/lib/clickhouse/data/test/mutation_simple/all_1_1_0/']",
E             -   'all_1_1_0_2',
E             -   '/var/lib/clickhouse/data/test/mutation_simple/all_1_1_0_2/',
E             -   'all',
E             -   '1'],
E               ]

test_system_merges/test.py:205: AssertionError
---------------------------- Captured stdout setup -----------------------------
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml
---------------------------- Captured stderr setup -----------------------------
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
------------------------------ Captured log setup ------------------------------
2023-12-28 11:43:58 [ 368 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:105, run_and_check)
2023-12-28 11:43:58 [ 368 ] DEBUG : Stdout:1 (cluster.py:113, run_and_check)
2023-12-28 11:43:58 [ 368 ] DEBUG : No running containers (conftest.py:60, cleanup_environment)
2023-12-28 11:43:58 [ 368 ] DEBUG : Pruning Docker networks (conftest.py:62, cleanup_environment)
2023-12-28 11:43:58 [ 368 ] DEBUG : Command:['docker network prune --force'] (cluster.py:105, run_and_check)
2023-12-28 11:43:58 [ 368 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:105, run_and_check)
2023-12-28 11:43:58 [ 368 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:113, run_and_check)
2023-12-28 11:43:58 [ 368 ] INFO : Running tests in /ClickHouse/tests/integration/test_system_merges/test.py (cluster.py:2637, start)
2023-12-28 11:43:58 [ 368 ] DEBUG : Cluster start called. is_up=False (cluster.py:2644, start)
2023-12-28 11:43:58 [ 368 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 39
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dd1e886e55dd695541fdcd67420c2888645a495
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-79-generic
 Operating System: Ubuntu 22.04.3 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a684db6f17f7
 ID: c4b50cd9-6b6a-42a0-a4c3-35114887c38f
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2023-12-28 11:43:58 [ 368 ] DEBUG : Docker networks for project roottestsystemmerges are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2023-12-28 11:43:58 [ 368 ] DEBUG : Docker containers for project roottestsystemmerges are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2023-12-28 11:43:58 [ 368 ] DEBUG : Docker volumes for project roottestsystemmerges are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2023-12-28 11:43:58 [ 368 ] DEBUG : Cleanup called (cluster.py:754, cleanup)
2023-12-28 11:43:58 [ 368 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 39
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dd1e886e55dd695541fdcd67420c2888645a495
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-79-generic
 Operating System: Ubuntu 22.04.3 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a684db6f17f7
 ID: c4b50cd9-6b6a-42a0-a4c3-35114887c38f
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2023-12-28 11:43:58 [ 368 ] DEBUG : Docker networks for project roottestsystemmerges are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2023-12-28 11:43:58 [ 368 ] DEBUG : Docker containers for project roottestsystemmerges are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2023-12-28 11:43:58 [ 368 ] DEBUG : Docker volumes for project roottestsystemmerges are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2023-12-28 11:43:58 [ 368 ] DEBUG : Command:docker container list --all --filter name='^/roottestsystemmerges_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:105, run_and_check)
2023-12-28 11:43:58 [ 368 ] DEBUG : Unstopped containers: {} (cluster.py:768, cleanup)
2023-12-28 11:43:58 [ 368 ] DEBUG : No running containers for project: roottestsystemmerges (cluster.py:782, cleanup)
2023-12-28 11:43:58 [ 368 ] DEBUG : Trying to prune unused networks... (cluster.py:788, cleanup)
2023-12-28 11:43:58 [ 368 ] DEBUG : Trying to prune unused images... (cluster.py:804, cleanup)
2023-12-28 11:43:58 [ 368 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:105, run_and_check)
2023-12-28 11:43:58 [ 368 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:113, run_and_check)
2023-12-28 11:43:58 [ 368 ] DEBUG : Images pruned (cluster.py:807, cleanup)
2023-12-28 11:43:58 [ 368 ] DEBUG : Trying to prune unused volumes... (cluster.py:813, cleanup)
2023-12-28 11:43:58 [ 368 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:105, run_and_check)
2023-12-28 11:43:58 [ 368 ] DEBUG : Stdout:2 (cluster.py:113, run_and_check)
2023-12-28 11:43:58 [ 368 ] DEBUG : Setup directory for instance: node1 (cluster.py:2657, start)
2023-12-28 11:43:58 [ 368 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4307, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Create directory for common tests configuration (cluster.py:4312, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Copy common configuration from helpers (cluster.py:4332, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Generate and write macros file (cluster.py:4351, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_system_merges/configs/logs_config.xml'] to /ClickHouse/tests/integration/test_system_merges/_instances_1/node1/configs/config.d (cluster.py:4381, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_system_merges/_instances_1/node1/database (cluster.py:4398, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_system_merges/_instances_1/node1/logs (cluster.py:4409, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4485, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Setup directory for instance: node2 (cluster.py:2657, start)
2023-12-28 11:43:58 [ 368 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4307, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Create directory for common tests configuration (cluster.py:4312, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Copy common configuration from helpers (cluster.py:4332, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Generate and write macros file (cluster.py:4351, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_system_merges/configs/logs_config.xml'] to /ClickHouse/tests/integration/test_system_merges/_instances_1/node2/configs/config.d (cluster.py:4381, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_system_merges/_instances_1/node2/database (cluster.py:4398, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_system_merges/_instances_1/node2/logs (cluster.py:4409, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4485, create_dir)
2023-12-28 11:43:58 [ 368 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/coordination'} stored in /ClickHouse/tests/integration/test_system_merges/_instances_1/.env (cluster.py:78, _create_env_file)
2023-12-28 11:43:58 [ 368 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file)
2023-12-28 11:43:58 [ 368 ] DEBUG : No config file found (config.py:28, find_config_file)
2023-12-28 11:43:58 [ 368 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file)
2023-12-28 11:43:58 [ 368 ] DEBUG : No config file found (config.py:28, find_config_file)
2023-12-28 11:43:58 [ 368 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 824 (connectionpool.py:546, _make_request)
2023-12-28 11:43:58 [ 368 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/.env', '--project-name', 'roottestsystemmerges', '--file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/node1/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/node2/docker-compose.yml', 'pull'] (cluster.py:105, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo1  ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo3  ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo2  ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node2 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo3  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo1  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo3  ... digest: sha256:21141428d3ff65738f... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo3  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo3  ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo2  ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node2 ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo1  ... digest: sha256:21141428d3ff65738f... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo1  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo1  ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo2  ... digest: sha256:21141428d3ff65738f... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo2  ... status: image is up to date for a... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node1 ... pulling from altinityinfra/integr... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling zoo2  ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node2 ... digest: sha256:21141428d3ff65738f... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node2 ... status: image is up to date for a... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node2 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node1 ... digest: sha256:21141428d3ff65738f... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node1 ... status: image is up to date for a... (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Stderr:Pulling node1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:10 [ 368 ] DEBUG : Setup ZooKeeper (cluster.py:2699, start)
2023-12-28 11:44:10 [ 368 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/log', '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/config', '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/log', '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/config', '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/coordination', '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/log', '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/config', '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/coordination'] (cluster.py:2700, start)
2023-12-28 11:44:10 [ 368 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/.env', '--project-name', 'roottestsystemmerges', '--file', '/compose/docker_compose_keeper.yml', '--verbose', 'up', '-d'] (cluster.py:105, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /compose/docker_compose_keeper.yml (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:docker-py version: <module 'docker.version' from '/usr/local/lib/python3.10/dist-packages/docker/version.py'> (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'amd64', 'BuildTime': '2023-05-05T21:18:13.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-79-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': '1.6.26', 'Details': {'GitCommit': '3dd1e886e55dd695541fdcd67420c2888645a495'}}, {'Name': 'runc', 'Version': '1.1.10', 'Details': {'GitCommit': 'v1.1.10-0-g18a0cb0'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=amd64, KernelVersion=5.15.0-79-generic, BuildTime=2023-05-05T21:18:13.000000000+00:00 (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('roottestsystemmerges_default') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'x86_64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '3dd1e886e55dd695541fdcd67420c2888645a495', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:                      'ID': '3dd1e886e55dd695541fdcd67420c2888645a495'}, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Containers': 0, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('roottestsystemmerges_default') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.network.ensure: Creating network "roottestsystemmerges_default" with the default driver (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network <- (name='roottestsystemmerges_default', driver=None, options=None, ipam=None, internal=False, enable_ipv6=False, labels={'com.docker.compose.project': 'roottestsystemmerges', 'com.docker.compose.network': 'default', 'com.docker.compose.version': '1.29.2'}, attachable=True, check_duplicate=True) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network -> {'Id': '6fe4b6a2a2bf8fe597068c455209590c65e40b1d3fc4e3abaa050a5dcb3083c8', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Warning': ''} (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {<Service: zoo3>, <Service: zoo1>, <Service: zoo2>} (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for <Service: zoo3> (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for <Service: zoo1> (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for <Service: zoo2> (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_zoo1_1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestsystemmerges', service='zoo1', number=1)} (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestsystemmerges', service='zoo1', number=1) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=roottestsystemmerges', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_zoo3_1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestsystemmerges', service='zoo3', number=1)} (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestsystemmerges', service='zoo3', number=1) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_zoo2_1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='roottestsystemmerges', service='zoo2', number=1)} (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='roottestsystemmerges', service='zoo2', number=1) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 6a0e59dc448ee20384e91fd0e79e3817dd4dbac182c73d94ac16d8b101250198 (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestsystemmerges_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Links': [], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:             'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/config', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:             'Target': '/etc/clickhouse-keeper', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:             'Type': 'bind'}, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            {'ReadOnly': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config1.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc', user='0', volumes={}, name='roottestsystemmerges_zoo1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestsystemmerges', 'com.docker.compose.service': 'zoo1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/compose', 'com.docker.compose.project.config_files': '/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_system_merges/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '6a0e59dc448ee20384e91fd0e79e3817dd4dbac182c73d94ac16d8b101250198'}, host_config={'NetworkMode': 'roottestsystemmerges_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestsystemmerges_default': {'Aliases': ['zoo1'], 'IPAMConfig': {}}}}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 17b0500861317c6e989571cde5fec5e0b07df12c4e75a79be6bc866f8207af08 (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestsystemmerges_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Links': [], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:             'Source': '/clickhouse', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:             'Target': '/usr/bin/clickhouse-keeper', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:             'Type': 'bind'}, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            {'ReadOnly': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config3.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc', user='0', volumes={}, name='roottestsystemmerges_zoo3_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestsystemmerges', 'com.docker.compose.service': 'zoo3', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/compose', 'com.docker.compose.project.config_files': '/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_system_merges/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '17b0500861317c6e989571cde5fec5e0b07df12c4e75a79be6bc866f8207af08'}, host_config={'NetworkMode': 'roottestsystemmerges_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestsystemmerges_default': {'Aliases': ['zoo3'], 'IPAMConfig': {}}}}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Author': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Entrypoint': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 1901f4bff9fcde6a5a3d45c9ceb87277f29f621f6006912adab4858e2367b676 (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='roottestsystemmerges_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Links': [], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:             'Source': '/clickhouse', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:             'Target': '/usr/bin/clickhouse-keeper', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:             'Type': 'bind'}, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            {'ReadOnly': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config2.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:0-57762a45dd8d7579a37ae9643654cde6896312dc', user='0', volumes={}, name='roottestsystemmerges_zoo2_1', detach=True, environment=[], labels={'com.docker.compose.project': 'roottestsystemmerges', 'com.docker.compose.service': 'zoo2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/compose', 'com.docker.compose.project.config_files': '/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_system_merges/_instances_1/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '1901f4bff9fcde6a5a3d45c9ceb87277f29f621f6006912adab4858e2367b676'}, host_config={'NetworkMode': 'roottestsystemmerges_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper2/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_system_merges/_instances_1/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'roottestsystemmerges_default': {'Aliases': ['zoo2'], 'IPAMConfig': {}}}}) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '247790d92a0d22cc4eeade70a4e13567021116a4a250456c6cc323ce5e05241e', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('247790d92a0d22cc4eeade70a4e13567021116a4a250456c6cc323ce5e05241e') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:          '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:          '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:          '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('247790d92a0d22cc4eeade70a4e13567021116a4a250456c6cc323ce5e05241e', 'roottestsystemmerges_default') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('247790d92a0d22cc4eeade70a4e13567021116a4a250456c6cc323ce5e05241e', 'roottestsystemmerges_default', aliases=['zoo2', '247790d92a0d'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('247790d92a0d22cc4eeade70a4e13567021116a4a250456c6cc323ce5e05241e') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'a8bbfcdb49985c1f9ba0603a16d8f6aae4497854885b819011d387292c20b3d9', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('a8bbfcdb49985c1f9ba0603a16d8f6aae4497854885b819011d387292c20b3d9') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'e25eedd4e5db4102b75563e1d5b21151e49374483013e9ec46eb4675ed3f4bdc', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('e25eedd4e5db4102b75563e1d5b21151e49374483013e9ec46eb4675ed3f4bdc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:          '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:          '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:          '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('a8bbfcdb49985c1f9ba0603a16d8f6aae4497854885b819011d387292c20b3d9', 'roottestsystemmerges_default') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:          '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:          '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:          '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdin': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'AttachStdout': False, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Cmd': None, (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:            'Domainname': '', (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:... (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('e25eedd4e5db4102b75563e1d5b21151e49374483013e9ec46eb4675ed3f4bdc', 'roottestsystemmerges_default') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('a8bbfcdb49985c1f9ba0603a16d8f6aae4497854885b819011d387292c20b3d9', 'roottestsystemmerges_default', aliases=['zoo3', 'a8bbfcdb4998'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('e25eedd4e5db4102b75563e1d5b21151e49374483013e9ec46eb4675ed3f4bdc', 'roottestsystemmerges_default', aliases=['e25eedd4e5db', 'zoo1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('a8bbfcdb49985c1f9ba0603a16d8f6aae4497854885b819011d387292c20b3d9') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('e25eedd4e5db4102b75563e1d5b21151e49374483013e9ec46eb4675ed3f4bdc') (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestsystemmerges', service='zoo2', number=1) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_zoo2_1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: <Service: zoo2> (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestsystemmerges', service='zoo3', number=1) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_zoo3_1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: <Service: zoo3> (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='roottestsystemmerges', service='zoo1', number=1) (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_zoo1_1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: <Service: zoo1> (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:115, run_and_check)
2023-12-28 11:44:11 [ 368 ] DEBUG : Wait ZooKeeper to start (cluster.py:2337, wait_zookeeper_to_start)
2023-12-28 11:44:11 [ 368 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:1971, get_instance_ip)
2023-12-28 11:44:11 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestsystemmerges_zoo1_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:11 [ 368 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.1.4, port:2181, use_ssl:False (cluster.py:3120, get_kazoo_client)
2023-12-28 11:44:11 [ 368 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect)
2023-12-28 11:44:11 [ 368 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2023-12-28 11:44:11 [ 368 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect)
2023-12-28 11:44:11 [ 368 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2023-12-28 11:44:11 [ 368 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect)
2023-12-28 11:44:11 [ 368 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2023-12-28 11:44:11 [ 368 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect)
2023-12-28 11:44:11 [ 368 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2023-12-28 11:44:12 [ 368 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect)
2023-12-28 11:44:12 [ 368 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2023-12-28 11:44:13 [ 368 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect)
2023-12-28 11:44:13 [ 368 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt)
2023-12-28 11:44:15 [ 368 ] INFO : Connecting to 172.16.1.4(172.16.1.4):2181, use_ssl: False (connection.py:650, _connect)
2023-12-28 11:44:15 [ 368 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit)
2023-12-28 11:44:15 [ 368 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback)
2023-12-28 11:44:15 [ 368 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit)
2023-12-28 11:44:15 [ 368 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response)
2023-12-28 11:44:15 [ 368 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit)
2023-12-28 11:44:15 [ 368 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt)
2023-12-28 11:44:15 [ 368 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt)
2023-12-28 11:44:15 [ 368 ] INFO : Zookeeper connection lost (client.py:543, _session_callback)
2023-12-28 11:44:15 [ 368 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop)
2023-12-28 11:44:15 [ 368 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback)
2023-12-28 11:44:15 [ 368 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:1971, get_instance_ip)
2023-12-28 11:44:15 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestsystemmerges_zoo2_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:15 [ 368 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.1.2, port:2181, use_ssl:False (cluster.py:3120, get_kazoo_client)
2023-12-28 11:44:15 [ 368 ] INFO : Connecting to 172.16.1.2(172.16.1.2):2181, use_ssl: False (connection.py:650, _connect)
2023-12-28 11:44:15 [ 368 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit)
2023-12-28 11:44:15 [ 368 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback)
2023-12-28 11:44:15 [ 368 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit)
2023-12-28 11:44:15 [ 368 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response)
2023-12-28 11:44:15 [ 368 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit)
2023-12-28 11:44:15 [ 368 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt)
2023-12-28 11:44:15 [ 368 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt)
2023-12-28 11:44:15 [ 368 ] INFO : Zookeeper connection lost (client.py:543, _session_callback)
2023-12-28 11:44:15 [ 368 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop)
2023-12-28 11:44:15 [ 368 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback)
2023-12-28 11:44:15 [ 368 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:1971, get_instance_ip)
2023-12-28 11:44:15 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestsystemmerges_zoo3_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:15 [ 368 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.1.3, port:2181, use_ssl:False (cluster.py:3120, get_kazoo_client)
2023-12-28 11:44:15 [ 368 ] INFO : Connecting to 172.16.1.3(172.16.1.3):2181, use_ssl: False (connection.py:650, _connect)
2023-12-28 11:44:15 [ 368 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit)
2023-12-28 11:44:15 [ 368 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback)
2023-12-28 11:44:15 [ 368 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit)
2023-12-28 11:44:15 [ 368 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response)
2023-12-28 11:44:15 [ 368 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit)
2023-12-28 11:44:15 [ 368 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt)
2023-12-28 11:44:15 [ 368 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt)
2023-12-28 11:44:15 [ 368 ] INFO : Zookeeper connection lost (client.py:543, _session_callback)
2023-12-28 11:44:15 [ 368 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop)
2023-12-28 11:44:15 [ 368 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback)
2023-12-28 11:44:15 [ 368 ] DEBUG : All instances of ZooKeeper started (cluster.py:2345, wait_zookeeper_to_start)
2023-12-28 11:44:15 [ 368 ] DEBUG : ('Trying to create ClickHouse instance by command %s', 'docker-compose --env-file /ClickHouse/tests/integration/test_system_merges/_instances_1/.env --project-name roottestsystemmerges --file /ClickHouse/tests/integration/test_system_merges/_instances_1/node1/docker-compose.yml --file /compose/docker_compose_keeper.yml --file /ClickHouse/tests/integration/test_system_merges/_instances_1/node2/docker-compose.yml up -d --no-recreate') (cluster.py:2980, start)
2023-12-28 11:44:15 [ 368 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/.env', '--project-name', 'roottestsystemmerges', '--file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/node1/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/node2/docker-compose.yml', 'up', '-d', '--no-recreate'] (cluster.py:105, run_and_check)
2023-12-28 11:44:16 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_node1_1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:16 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_node2_1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:16 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_node1_1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:16 [ 368 ] DEBUG : Stderr:Creating roottestsystemmerges_node2_1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:16 [ 368 ] DEBUG : ClickHouse instance created (cluster.py:2988, start)
2023-12-28 11:44:16 [ 368 ] DEBUG : get_instance_ip instance_name=node1 (cluster.py:1971, get_instance_ip)
2023-12-28 11:44:16 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestsystemmerges_node1_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:16 [ 368 ] DEBUG : Waiting for ClickHouse start in node1, ip: 172.16.1.6... (cluster.py:2995, start)
2023-12-28 11:44:16 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestsystemmerges_node1_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:16 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/65580e7f3bc6fe55042a0f565b01561f7c87eada1dbb93e74b5e62c025972198/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:16 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/65580e7f3bc6fe55042a0f565b01561f7c87eada1dbb93e74b5e62c025972198/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:16 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/65580e7f3bc6fe55042a0f565b01561f7c87eada1dbb93e74b5e62c025972198/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:16 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/65580e7f3bc6fe55042a0f565b01561f7c87eada1dbb93e74b5e62c025972198/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:16 [ 368 ] DEBUG : ClickHouse node1 started (cluster.py:2999, start)
2023-12-28 11:44:16 [ 368 ] DEBUG : get_instance_ip instance_name=node2 (cluster.py:1971, get_instance_ip)
2023-12-28 11:44:16 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestsystemmerges_node2_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:16 [ 368 ] DEBUG : Waiting for ClickHouse start in node2, ip: 172.16.1.5... (cluster.py:2995, start)
2023-12-28 11:44:16 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestsystemmerges_node2_1/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:16 [ 368 ] DEBUG : http://localhost:None "GET /v1.42/containers/4c83568269c48a15bcb0890a1060e57968752c46d22f0d164b300e296af1aa19/json HTTP/1.1" 200 None (connectionpool.py:546, _make_request)
2023-12-28 11:44:16 [ 368 ] DEBUG : ClickHouse node2 started (cluster.py:2999, start)
2023-12-28 11:44:16 [ 368 ] DEBUG : Executing query CREATE DATABASE test ENGINE=Ordinary on node1 (cluster.py:3423, query)
2023-12-28 11:44:16 [ 368 ] DEBUG : Executing query CREATE DATABASE test ENGINE=Ordinary on node2 (cluster.py:3423, query)
------------------------------ Captured log call -------------------------------
2023-12-28 11:44:16 [ 368 ] DEBUG : Executing query create table test.mutation_simple (a Int64) engine=MergeTree() order by tuple() on node1 (cluster.py:3423, query)
2023-12-28 11:44:16 [ 368 ] DEBUG : Executing query INSERT INTO test.mutation_simple VALUES (1), (2), (3) on node1 (cluster.py:3423, query)
2023-12-28 11:44:16 [ 368 ] DEBUG : Executing query ALTER TABLE test.mutation_simple UPDATE a = 42 WHERE sleep(9) = 0 on node1 (cluster.py:3423, query)
2023-12-28 11:44:16 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node1 (cluster.py:3423, query)
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query 
            SELECT database, table, num_parts, source_part_names, source_part_paths, result_part_name, result_part_path, partition_id, is_mutation
                FROM system.merges
                WHERE table = 'mutation_simple'
         on node1 (cluster.py:3423, query)
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query DROP TABLE test.mutation_simple on node1 (cluster.py:3423, query)
_______________________ test_mutation_simple[replicated] _______________________
[gw1] linux -- Python 3.10.12 /usr/bin/python3

started_cluster = <helpers.cluster.ClickHouseCluster object at 0x7f2592952a40>
replicated = 'replicated'

    @pytest.mark.parametrize("replicated", ["", "replicated"])
    def test_mutation_simple(started_cluster, replicated):
        clickhouse_path = "/var/lib/clickhouse"
        db_name = "test"
        table_name = "mutation_simple"
        name = db_name + "." + table_name
        table_path = "data/" + db_name + "/" + table_name
        nodes = [node1, node2] if replicated else [node1]
        engine = (
            "ReplicatedMergeTree('/clickhouse/test_mutation_simple', '{replica}')"
            if replicated
            else "MergeTree()"
        )
        node_check = nodes[-1]
        starting_block = 0 if replicated else 1
    
        try:
            for node in nodes:
                node.query(
                    f"create table {name} (a Int64) engine={engine} order by tuple()"
                )
    
            node1.query(f"INSERT INTO {name} VALUES (1), (2), (3)")
    
            part = "all_{}_{}_0".format(starting_block, starting_block)
            result_part = "all_{}_{}_0_{}".format(
                starting_block, starting_block, starting_block + 1
            )
    
            # ALTER will sleep for 3s * 3 (rows) = 9s
            def alter():
                node1.query(
                    f"ALTER TABLE {name} UPDATE a = 42 WHERE sleep(9) = 0",
                    settings=settings,
                )
    
            t = threading.Thread(target=alter)
            t.start()
    
            # Wait for the mutation to actually start
>           assert_eq_with_retry(
                node_check,
                f"select count() from system.merges where table='{table_name}'",
                "1\n",
                retry_count=30,
                sleep_time=0.1,
            )

test_system_merges/test.py:197: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

instance = <helpers.cluster.ClickHouseInstance object at 0x7f2592952d40>
query = "select count() from system.merges where table='mutation_simple'"
expectation = '1\n', retry_count = 30, sleep_time = 0.1, stdin = None
timeout = None, settings = None, user = None, ignore_error = False
get_result = <function <lambda> at 0x7f2593f0c820>

    def assert_eq_with_retry(
        instance,
        query,
        expectation,
        retry_count=20,
        sleep_time=0.5,
        stdin=None,
        timeout=None,
        settings=None,
        user=None,
        ignore_error=False,
        get_result=lambda x: x,
    ):
        expectation_tsv = TSV(expectation)
        for i in range(retry_count):
            try:
                if (
                    TSV(
                        get_result(
                            instance.query(
                                query,
                                user=user,
                                stdin=stdin,
                                timeout=timeout,
                                settings=settings,
                                ignore_error=ignore_error,
                            )
                        )
                    )
                    == expectation_tsv
                ):
                    break
                time.sleep(sleep_time)
            except Exception as ex:
                logging.exception(f"assert_eq_with_retry retry {i+1} exception {ex}")
                time.sleep(sleep_time)
        else:
            val = TSV(
                get_result(
                    instance.query(
                        query,
                        user=user,
                        stdin=stdin,
                        timeout=timeout,
                        settings=settings,
                        ignore_error=ignore_error,
                    )
                )
            )
            if expectation_tsv != val:
>               raise AssertionError(
                    "'{}' != '{}'\n{}".format(
                        expectation_tsv,
                        val,
                        "\n".join(expectation_tsv.diff(val, n1="expectation", n2="query")),
                    )
                )
E               AssertionError: '1' != '0'
E               @@ -1 +1 @@
E               -1
E               +0

helpers/test_tools.py:114: AssertionError
------------------------------ Captured log call -------------------------------
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query create table test.mutation_simple (a Int64) engine=ReplicatedMergeTree('/clickhouse/test_mutation_simple', '{replica}') order by tuple() on node1 (cluster.py:3423, query)
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query create table test.mutation_simple (a Int64) engine=ReplicatedMergeTree('/clickhouse/test_mutation_simple', '{replica}') order by tuple() on node2 (cluster.py:3423, query)
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query INSERT INTO test.mutation_simple VALUES (1), (2), (3) on node1 (cluster.py:3423, query)
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query ALTER TABLE test.mutation_simple UPDATE a = 42 WHERE sleep(9) = 0 on node1 (cluster.py:3423, query)
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:17 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:18 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:18 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:18 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:18 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:18 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:18 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:19 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:19 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:19 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:19 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:19 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:19 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:20 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:20 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:20 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:20 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:20 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:20 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:21 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:21 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:21 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:21 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:21 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:21 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:22 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:22 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:22 [ 368 ] DEBUG : Executing query select count() from system.merges where table='mutation_simple' on node2 (cluster.py:3423, query)
2023-12-28 11:44:22 [ 368 ] DEBUG : Executing query DROP TABLE test.mutation_simple on node1 (cluster.py:3423, query)
2023-12-28 11:44:22 [ 368 ] DEBUG : Executing query DROP TABLE test.mutation_simple on node2 (cluster.py:3423, query)
--------------------------- Captured stderr teardown ---------------------------
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
---------------------------- Captured log teardown -----------------------------
2023-12-28 11:44:23 [ 368 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/.env', '--project-name', 'roottestsystemmerges', '--file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/node1/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/node2/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:105, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_node1_1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_node2_1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_zoo1_1  ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_zoo3_1  ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_zoo2_1  ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_node1_1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_node2_1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_zoo3_1  ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_zoo1_1  ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Stopping roottestsystemmerges_zoo2_1  ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_system_merges/_instances_1/node1/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_system_merges/_instances_1/node1/logs/stderr.log* || true'] (cluster.py:105, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_system_merges/_instances_1/node2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_system_merges/_instances_1/node2/logs/stderr.log* || true'] (cluster.py:105, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/.env', '--project-name', 'roottestsystemmerges', '--file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/node1/docker-compose.yml', '--file', '/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/test_system_merges/_instances_1/node2/docker-compose.yml', 'down', '--volumes'] (cluster.py:105, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_node1_1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_node2_1 ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_zoo1_1  ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_zoo3_1  ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_zoo2_1  ...  (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_zoo2_1  ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_node2_1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_node1_1 ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_zoo3_1  ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing roottestsystemmerges_zoo1_1  ... done (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stderr:Removing network roottestsystemmerges_default (cluster.py:115, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Cleanup called (cluster.py:754, cleanup)
2023-12-28 11:44:26 [ 368 ] DEBUG : !!! Docker info: Client: Docker Engine - Community
 Version:    24.0.7
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 39
 Server Version: 23.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dd1e886e55dd695541fdcd67420c2888645a495
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-79-generic
 Operating System: Ubuntu 22.04.3 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 30.6GiB
 Name: a684db6f17f7
 ID: c4b50cd9-6b6a-42a0-a4c3-35114887c38f
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  65.108.242.32:5000
  127.0.0.0/8
 Registry Mirrors:
  http://65.108.242.32:5000/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/12, Size: 24

 (cluster.py:722, print_all_docker_pieces)
2023-12-28 11:44:26 [ 368 ] DEBUG : Docker networks for project roottestsystemmerges are NETWORK ID   NAME      DRIVER    SCOPE
 (cluster.py:733, print_all_docker_pieces)
2023-12-28 11:44:26 [ 368 ] DEBUG : Docker containers for project roottestsystemmerges are CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
 (cluster.py:741, print_all_docker_pieces)
2023-12-28 11:44:26 [ 368 ] DEBUG : Docker volumes for project roottestsystemmerges are DRIVER    VOLUME NAME
 (cluster.py:749, print_all_docker_pieces)
2023-12-28 11:44:26 [ 368 ] DEBUG : Command:docker container list --all --filter name='^/roottestsystemmerges_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:105, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Unstopped containers: {} (cluster.py:768, cleanup)
2023-12-28 11:44:26 [ 368 ] DEBUG : No running containers for project: roottestsystemmerges (cluster.py:782, cleanup)
2023-12-28 11:44:26 [ 368 ] DEBUG : Trying to prune unused networks... (cluster.py:788, cleanup)
2023-12-28 11:44:26 [ 368 ] DEBUG : Trying to prune unused images... (cluster.py:804, cleanup)
2023-12-28 11:44:26 [ 368 ] DEBUG : Command:['docker', 'image', 'prune', '-f'] (cluster.py:105, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stdout:Total reclaimed space: 0B (cluster.py:113, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Images pruned (cluster.py:807, cleanup)
2023-12-28 11:44:26 [ 368 ] DEBUG : Trying to prune unused volumes... (cluster.py:813, cleanup)
2023-12-28 11:44:26 [ 368 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:105, run_and_check)
2023-12-28 11:44:26 [ 368 ] DEBUG : Stdout:2 (cluster.py:113, run_and_check)
=============================== warnings summary ===============================
test_system_merges/test.py::test_mutation_simple[]
  /usr/local/lib/python3.10/dist-packages/_pytest/threadexception.py:73: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-10 (alter)
  
  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/ClickHouse/tests/integration/test_system_merges/test.py", line 188, in alter
      node1.query(
    File "/ClickHouse/tests/integration/helpers/cluster.py", line 3424, in query
      return self.client.query(
    File "/ClickHouse/tests/integration/helpers/client.py", line 36, in wrap
      return func(self, *args, **kwargs)
    File "/ClickHouse/tests/integration/helpers/client.py", line 74, in query
      ).get_answer()
    File "/ClickHouse/tests/integration/helpers/client.py", line 239, in get_answer
      raise QueryRuntimeException(
  helpers.client.QueryRuntimeException: Client failed! Return code: 85, stderr: Received exception from server (version 23.8.8):
  Code: 341. DB::Exception: Received from 172.16.1.6:9000. DB::Exception: Exception happened during execution of mutation 'mutation_2.txt' with part 'all_1_1_0' reason: 'Code: 160. DB::Exception: The maximum sleep time is 3000000 microseconds. Requested: 9: while executing 'FUNCTION sleep(9 :: 0) -> sleep(9) UInt8 : 3'. (TOO_SLOW) (version 23.8.8.21.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. Stack trace:
  
  0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c644ed7 in /usr/bin/clickhouse
  1. DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x0000000012d8cc13 in /usr/bin/clickhouse
  2. DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x0000000012d8cabb in /usr/bin/clickhouse
  3. DB::StorageMergeTree::waitForMutation(long, String const&, bool) @ 0x0000000012f615bd in /usr/bin/clickhouse
  4. DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000012f62400 in /usr/bin/clickhouse
  5. DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000011dae605 in /usr/bin/clickhouse
  6. DB::InterpreterAlterQuery::execute() @ 0x0000000011dacad0 in /usr/bin/clickhouse
  7. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122e2d15 in /usr/bin/clickhouse
  8. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122de475 in /usr/bin/clickhouse
  9. DB::TCPHandler::runImpl() @ 0x0000000013155799 in /usr/bin/clickhouse
  10. DB::TCPHandler::run() @ 0x0000000013167b79 in /usr/bin/clickhouse
  11. Poco::Net::TCPServerConnection::start() @ 0x0000000015b5e154 in /usr/bin/clickhouse
  12. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b5f351 in /usr/bin/clickhouse
  13. Poco::PooledThread::run() @ 0x0000000015c95b87 in /usr/bin/clickhouse
  14. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c93e5c in /usr/bin/clickhouse
  15. ? @ 0x00007f0f0a4e2ac3 in ?
  16. ? @ 0x00007f0f0a574660 in ?
  . (UNFINISHED)
  (query: ALTER TABLE test.mutation_simple UPDATE a = 42 WHERE sleep(9) = 0)
  
  
    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

test_system_merges/test.py::test_mutation_simple[replicated]
  /usr/local/lib/python3.10/dist-packages/_pytest/threadexception.py:73: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-11 (alter)
  
  Traceback (most recent call last):
    File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
      self.run()
    File "/usr/lib/python3.10/threading.py", line 953, in run
      self._target(*self._args, **self._kwargs)
    File "/ClickHouse/tests/integration/test_system_merges/test.py", line 188, in alter
      node1.query(
    File "/ClickHouse/tests/integration/helpers/cluster.py", line 3424, in query
      return self.client.query(
    File "/ClickHouse/tests/integration/helpers/client.py", line 36, in wrap
      return func(self, *args, **kwargs)
    File "/ClickHouse/tests/integration/helpers/client.py", line 74, in query
      ).get_answer()
    File "/ClickHouse/tests/integration/helpers/client.py", line 239, in get_answer
      raise QueryRuntimeException(
  helpers.client.QueryRuntimeException: Client failed! Return code: 85, stderr: Received exception from server (version 23.8.8):
  Code: 341. DB::Exception: Received from 172.16.1.6:9000. DB::Exception: Exception happened during execution of mutation '0000000000' with part 'all_0_0_0' reason: 'Code: 160. DB::Exception: The maximum sleep time is 3000000 microseconds. Requested: 9: while executing 'FUNCTION sleep(9 :: 0) -> sleep(9) UInt8 : 3'. (TOO_SLOW) (version 23.8.8.21.altinitystable (altinity build))'. This error maybe retryable or not. In case of unretryable error, mutation can be killed with KILL MUTATION query. Stack trace:
  
  0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c644ed7 in /usr/bin/clickhouse
  1. DB::Exception::Exception<char const*, String, String&, String&>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String&>::type, std::type_identity<String&>::type>, char const*&&, String&&, String&, String&) @ 0x0000000012d8cc13 in /usr/bin/clickhouse
  2. DB::checkMutationStatus(std::optional<DB::MergeTreeMutationStatus>&, std::set<String, std::less<String>, std::allocator<String>> const&) @ 0x0000000012d8cabb in /usr/bin/clickhouse
  3. DB::StorageReplicatedMergeTree::waitMutationToFinishOnReplicas(std::vector<String, std::allocator<String>> const&, String const&) const @ 0x000000001277c1f8 in /usr/bin/clickhouse
  4. DB::StorageReplicatedMergeTree::waitMutation(String const&, unsigned long) const @ 0x0000000012819aa1 in /usr/bin/clickhouse
  5. DB::StorageReplicatedMergeTree::mutate(DB::MutationCommands const&, std::shared_ptr<DB::Context const>) @ 0x0000000012841f83 in /usr/bin/clickhouse
  6. DB::InterpreterAlterQuery::executeToTable(DB::ASTAlterQuery const&) @ 0x0000000011dae605 in /usr/bin/clickhouse
  7. DB::InterpreterAlterQuery::execute() @ 0x0000000011dacad0 in /usr/bin/clickhouse
  8. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x00000000122e2d15 in /usr/bin/clickhouse
  9. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum) @ 0x00000000122de475 in /usr/bin/clickhouse
  10. DB::TCPHandler::runImpl() @ 0x0000000013155799 in /usr/bin/clickhouse
  11. DB::TCPHandler::run() @ 0x0000000013167b79 in /usr/bin/clickhouse
  12. Poco::Net::TCPServerConnection::start() @ 0x0000000015b5e154 in /usr/bin/clickhouse
  13. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b5f351 in /usr/bin/clickhouse
  14. Poco::PooledThread::run() @ 0x0000000015c95b87 in /usr/bin/clickhouse
  15. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c93e5c in /usr/bin/clickhouse
  16. ? @ 0x00007f0f0a4e2ac3 in ?
  17. ? @ 0x00007f0f0a574660 in ?
  . (UNFINISHED)
  (query: ALTER TABLE test.mutation_simple UPDATE a = 42 WHERE sleep(9) = 0)
  
  
    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================== slowest durations ===============================
18.36s setup    test_system_merges/test.py::test_mutation_simple[]
5.45s call     test_system_merges/test.py::test_mutation_simple[replicated]
4.24s teardown test_system_merges/test.py::test_mutation_simple[replicated]
0.35s call     test_system_merges/test.py::test_mutation_simple[]
0.00s teardown test_system_merges/test.py::test_mutation_simple[]
0.00s setup    test_system_merges/test.py::test_mutation_simple[replicated]
=========================== short test summary info ============================
FAILED test_system_merges/test.py::test_mutation_simple[] - assert [] == [['t...
FAILED test_system_merges/test.py::test_mutation_simple[replicated] - Asserti...
======================== 2 failed, 2 warnings in 29.42s ========================
Traceback (most recent call last):
  File "/home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration/./runner", line 455, in <module>
    subprocess.check_call(cmd, shell=True)
  File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'docker run   --rm --name clickhouse_integration_tests_hcivdn --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/_temp/test/git-repo-copy/src/Server/grpc_protos:/ClickHouse/src/Server/grpc_protos --volume=/run:/run/host:ro --mount type=bind,source=/home/ubuntu/_work/_temp/test/dockerd_volume_dir,target=/var/lib/docker -e DOCKER_HELPER_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_BASE_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_KERBERIZED_HADOOP_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_KERBEROS_KDC_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_MYSQL_JAVA_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_MYSQL_JS_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_MYSQL_PHP_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_NGINX_DAV_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=0-57762a45dd8d7579a37ae9643654cde6896312dc   -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600  -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 5 -rfEps --run-id=1 --color=no --durations=0 'test_system_merges/test.py::test_mutation_simple[]' 'test_system_merges/test.py::test_mutation_simple[replicated]'  -vvv" altinityinfra/integration-tests-runner:0-57762a45dd8d7579a37ae9643654cde6896312dc ' returned non-zero exit status 1.